DISTRIBUTIONS ON PARTITIONS, POINT 
PROCESSES, AND THE HYPERGEOMETRIC KERNEL 

Alexei Borodin and Grigori Olshanski 

Abstract. We study a 3-parametric family of stochastic point processes on the 
one-dimensional lattice originated from a remarkable family of representations of the 
infinite symmetric group. We prove that the correlation functions of the processes are 
given by determinantal formulas with a certain kernel. The kernel can be expressed 
through the Gauss hypergeometric function; we call it the hypergeometric kernel. 

In a scaling limit our processes approximate the processes describing the decom- 
position of representations mentioned above into irreducibles. As we showed before, 
see math.RT/9810015, the correlation functions of these limit processes also have 
determinantal form with so-called Whittaker kernel. We show that the scaling limit 
of the hypergeometric kernel is the Whittaker kernel. 

The integral operator corresponding to the Whittaker kernel is an integrable op- 
erator as defined by Its, Izergin, Korepin, and Slavnov. We argue that the hypergeo- 
metric kernel can be considered as a kernel defining a 'discrete integrable operator'. 

We also show that the hypergeometric kernel degenerates for certain values of 
parameters to the Christoffel-Darboux kernel for Meixner orthogonal polynomials. 
This fact is parallel to the degeneration of the Whittaker kernel to the Christoffel- 
Darboux kernel for Laguerre polynomials. 
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§0. Introduction 

Let be the set of Young diagrams with n boxes and Y = Yq U Yi U Y2 U . . . 
be the set of all Young diagrams. In this paper we study a remarkable family of 
probability distributions on Y„, n = 0, 1, 2, . . . . 

The whole picture depends on 2 parameters z and z' which satisfy certain con- 
ditions, see §1. For each pair {z, z') we have a probability distribution on every Y^, 



n = 0,l,2,...,we denote it by M^^^,. Its value on a Young diagram A e with 
Frobenius coordinates (pi, ■ ■ ■ ,Pd \ Qi, ■ • • , Qd) has the form 



Mi5(A) 



ni 



[zz'). 



{zz') 



/\d 



n 

i=l 



Pilpilqilqil 



1 



Pi + Qj + 1 



(0.1) 



where {a)k stands for a{a + 1) ■ ■ ■ (a + — 1). 

The distributions M^"^, have a representation-theoretic meaning. Let Sn be the 
symmetric group of degree n, 5'(oo) be the union of the groups Sn, and for A G Y„, 
let denote the irreducible character of 5'^ corresponding to A. According to 
[KOV], there exists a central positive definite function x^^'^ ^ on S{oo) such that, 
for any n, its restriction to is 



|A|=n 



Moreover, the unitary representation T^^'^ -* corresponding to x^^'^ admits a nice 
geometric description (at least for z' = z), see [KOV]. This representation-theoretic 
aspect was the original motivation of our study but in the present paper we do not 
discuss it (see [P.I]). 

Let us associate to a Young diagram A = (pi, . . . ,pd\qi, ■ ■ ■ , qd) a set of 2d points 
in Z' = Z + I as follows: 

A = {pi,...,pd\qi,...,qd) ^ {pi + l,...,Pd + - .., -gd - |}- 

Then every probability measure on Y„ provides a probability measure on the set 
of all point configurations in Z' with equal number of points in Z^ = Z' n and 
Z'_ = Z' n R_ and such that the total sum of absolute values of coordinates is equal 
to n. 

Next, having a distribution on each Y„, we can mix them using a distribution 
on the set {0, 1, 2, ... } of indices n, then we get a probability distribution on Y. 

Thus, we get a probability measure on the set of all point configurations in Z' 
with equal number of points in Z^ and Z'_. According to standard terminology, 
we can say that we defined a point process on Z'. 

Following a certain analogy with statistical physics, one can call the resulting 
object of the mixing procedure the grand canonical ensemble, see [V] . 

For our special distributions (0.1) we choose the mixing distribution to be the 
negative binomial distribution 

Prob{n} = (l-O*^r, t = zz', (0.2) 

where ^ G (0, 1) is an additional parameter. (This choice is explained by willing to 
remove the factor -^y- from the RHS of (0.1).) We shall denote by Vz,z',£, the point 
process on Z' thus obtained. 
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The main result of this paper is the exphcit computation of the correlation 
functions of Vz,z',^- It turns out that they are given by determinantal formulas 

Pn{xi, ...,Xn)^ det[K{Xi, Xj)]lj^i 

with a certain kernel K{x, y) on Z'. This kernel can be expressed through the Gauss 
hypergeometric function. We call it the hypergeometric kernel. 

Due to the representation theoretic origin of our problem, the distributions M^J^^, 
have a number of additional properties. In particular, as n — > oo, they converge 
to a probability measure on a certain limit object called the 'Thoma simplex', see 
[KOO] and §5 below. ^ 

In terms of point processes, this implies that after an appropriate scaling the 
point processes Vz,z',£, will converge, as ^ /" 1, to a certain point process on R* = 
M \ {0} derived from the limit measure on the Thoma simplex (we shall give a 
rigorous proof of this result in our next paper) . 

This limit process has been thoroughly studied in our previous papers [P.I] - 
[P.V]^. Its correlation functions also have determinantal form with so-called Whit- 
taker kernel, see [P-IV]. In §5 we show directly that the scaling limit of the hyper- 
geometric kernel is the Whittaker kernel. 

The integral operator defined by the Whittaker kernel belongs to the class of 
integrable operators as defined by Its, Izergin, Korepin, Slavnov [IIKS]. We show 
that the operator corresponding to the hypergeometric kernel can be considered as 
an example of a 'discrete integrable operator'. 

A. Okounkov pointed out that important information can be obtained from con- 
sideration of another degeneration of the point process introduced above. Assume 
that — > oo and ^ = ^ = ^ ^ where 77 > is fixed. Then the mixing 
distribution (0.2) tends to the Poisson distribution with parameter r]: 

Probln} e ^ —r, 

(n) 

while M^J tends to the Plancherel distribution on Y„: 

Mi:?(A)^MH(A) = ^, (0.3) 

where dim A = x'^(e) is the dimension of the irreducible representation of the sym- 
metric group Sn corresponding to A. Thus, we get an explicit formula for the 
correlation functions of the process governed by the poissonized Plancherel distri- 
butions.^ 

This formula allows to prove certain important facts about Plancherel distri- 
butions, see [BOO]. In particular, we were able to prove the conjecture by Baik, 
Deift, and Johansson [BDJl, BDJ2] that the asymptotic behavior of Ai,A2,... 

■"■This is a kind of dual object to the infinite symmetric group, see [T], [VK], [KV]. The limit 
measure is, actually, a spectral measure for the decomposition of the representation T(^'^ ) into 
irreducibles, see [KOV]. 

survey of the results is given in [BOX]. 

^The limit relation (0.3) was known since the invention of the distributions see [KOV], 

but up to now it was not used. 
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with respect to the Plancherel distribution is governed by the Airy kernel [TW] 
and, therefore, coincides with that of the largest eigenvalues of a matrix from the 
Gaussian Unitary Ensemble. (Another approach to this conjecture can be found in 
[O].) 

The paper is organized as follows. In §1 we introduce our main object of interest 

- the point process Vz,z'4- In §2 we recall some generalities on determinantal 
point processes. The computation of the correlation functions of Vz,z',^ and the 
formulas for the hypergeometric kernel can be found in §3. In §4 we show that if 
one of the parameters z, z' is an integer, the hypergeometric kernel degenerates 
to the ChristoflFel-Darboiix kernel for the Meixner orthogonal polynomials on Z_|_.^ 
In §5 we discuss the scaling limit of the hypergeometric kernel. §6 explains the 
connection with integrable operators. §7 is an appendix, we give there proofs of 
certain identities involving the Gauss hypergeometric functions which are used in 
§3. 

Acknowledgements. We are grateful to P. Deift and N. A. Slavnov for consul- 
tations concerning the subject of §6. We also thank C. A. Tracy for telling us 
about Johansson's talk at MSRI and sending us a copy of the transparencies, and 
K. Johansson for further information about his work [J] . 

§1. Distributions on partitions. The grand canonical ensemble 

For n = 1, 2, . . . , let denote the set of partitions of n, which will be identified 
with Young diagrams with n boxes. We agree that Yq consists of a single element 

— the zero partition or the empty diagram 0. 

Given A e Y^, we write |A| = n and denote by d = d{X) the number of diagonal 
boxes in A. We shall use the Frobenius notation [Ma] 

A = {pi,...,pd\qi,...,qd)- 

Here pi — Xi — i is the number of boxes in the ith row of A on the right of the ith 
diagonal box; likewise, = A^ — z is the number of boxes in the ith column of A 
below the ith diagonal box (A' stands for the transposed diagram). 
Note that 

d 

Pi> ■■■>Pd>0, qi> ■■■> qd>0, ^(pi + gi + 1) = |A|. 

i=l 

The numbers Pi, qi are called the Frobenius coordinates of the diagram A. 

Throughout the paper we fix two complex parameters z, z' such that the numbers 
{z)k{/^'^k and (— 2;)fc(— are real and strictly positive for any k = 1, 2, ... . Here 
and below 

(a)fc = a(a + 1) . . . (a + /c - 1), (a)o = 1, 

denotes the Pochhammer symbol. 

The above assumption on z^ z' means that one of the following two conditions 
holds: 

• either z' = z and 2; e C \ Z 



After the present paper was completed we learned that the 'Meixner kernel' has also arisen 
in the recent work [J]. 
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• or z, z' eM. and there exists m e Z such that m < z, z' < m + 1. 
We set 

t = zz' 



and note that t > 0. 

For a Young diagram A let dim A denote the number of the standard Young 
tableaiix of shape A. Equivalently, dim A is the dimension of the irreducible repre- 
sentation (of the symmetric group of degree |A|) corresponding to A, see [Ma]. In 
the Frobenius notation, 



^ n {pi.-p.i){(ii-<ij] 

dim A _ i</:<j <(/ 

~Jxf^ n (Pi + qj + l) 

l<i,j<n 



= det 



Pi + Qj + 1 



l<i,j<d 



see, e.g., [P.I, Prop. 2.6]. 

We introduce a function on the Young diagrams depending on the parameters 
z, z' : 



Mz,z'W = 7^ n 



{z + l)p^{z' + l)p^{-z + l),,{-z' + 1),, dim^ A 
{t)n fJl Pilpilqilq: 



I 



lAI! 



|A|! 



i=l 



{z + l)p,iz' + l)p^{-z + l),,i-z' + l) 
Pi\pi\qi\qi\ 



det' 



Pi + qj + 1 



We agree that M^ ,j/(0) = 1. Thanks to our assumption on the parameters, 
M^,^,(A) > for aU A.' 



Proposition 1.1. For any n, 



M,,,,(A) = 1, 



A€Y„ 



SO that the restriction of M^^z' to is a probability distribution on Y^- 

(n) 

We shall denote this distribution by J, . 

Comments. This result is the starting point of our investigations. About its ori- 
gin and representation-theoretic significance, see [KOV]. Several direct proofs of 
the proposition are known. E.g., a simple proof is given in [Part I, §7]. About 
generalizations, see [K] , [B02] . Note that 

dim' A 

hm Mz^,>{X)^ 
\z\,\z'\^oo |A|! 

so that the limit form of the identity of the proposition is 



E 



dim' A 



which is well known. 



Let Y = Yq U Yi U . . . denote the set of all Young diagrams. Consider the 
negative binomial distribution on the nonnegative integers, which depends on t and 
the additional parameter ^, < ^ < 1: 

^U(n) = (l-0'^r, n = 0,l,.... 

For A e Y we set 

M,,,,,^(A) = M,,,,(A)7r,,^(|A|). 

By the construction, M^^^'.^IO is a probability distribution on Y, which can be 
viewed as a mixing of the finite distributions ilff "2 • Prom the formulas for 
and 'Kt.i we get an explicit expression for M^^z' 

M.,.,(A) = (i-eni^™t^ 

Following a certain analogy with models of statistical physics (cf. [V]) one may 
call {\ ,Mz^z' ,i) the grand canonical ensemble. 
Let Z' denote the set of half-integers, 

_3_113 1 

and let and Z'_ be the subsets of positive and negative half-integers, respectively. 
It will be sometimes convenient to identify both Z^ and Z'_ with Z+ = {0, 1, 2, . . . } 
by making use of the correspondence ±(A; + |) <-> A;, where A; e Z+. 

Denote by Conf(Z') the space of all finite subsets of Z' which will be called 
configurations. We define an embedding A i-^ X of the set Y of Young diagrams 
into the set Conf(Z') of configurations in Z' as follows: 

A = (pi,.. 1^1,.. ^ X = {pi + J,. \,-qi -\,...,-qd- \}. 

Under the identification Z' ~ Z_|_ U Z_|_, the map A i— > X is simply associating to A 
the collection of its Frobenius coordinates. The image of the map consists exactly 
of the configurations X with the property \X fl Z^| = |X n Z'_|. 

Under the embedding A i— > X the probability measure M^^z' ,s, ^ turns into a 
probability measure on the configurations in Z'. Following the conventional termi- 
nology, see [DVJ], we get a point process on Z'; let us denote it as Vz,z',£,- 

Our primary goal is to compute the correlation functions of this point process. 

§2. Determinantal point processes 

Let X be a countable set. Its finite subsets will be called configurations and 
denoted by the letters X, Y . The space of all configurations is denoted as Conf (X); 
this is a discrete space. In this section, by a point process on X we mean a map 
from a probability space to Conf(X).^ Let P be a point process on X. It induces a 



Pi + g,- + 1 



• (1-1) 



^Actually, such a definition is rather restricted but it suffices for our purpose. For general 
axiomatics of point processes, see [DVJ], [LI], [L2]. 
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probability distribution on Conf (X), which is a nonnegative function 7r{X) such that 
Tv{X) = 1. One may simply identify V and tt: then the underlying probability 
space is Conf(X) itself. 

We introduce a related function on Conf(X) as follows: 

p{x) = J2 <n 

That is, p{X) is the probability that the random configuration contains X. 

Consider the Hilbcrt space An operator L in £^(X) will be viewed as an 

infinite matrix L(,t, y) whose rows and columns are indexed by points of X. By Lx 
we denote the finite matrix of format X x X which is obtained from L by letting 

y range over X. 

Assume L is a trace class operator in X such that all the principal minors det Lx 
are real nonnegative numbers. We agree that detL0 = 1. We have 

det(l + L) = ^tr(A''L) = ^detLx 

n X 

(here A"L stands for the nth exterior power of L acting in the nth exterior power 
of the Hilbert space £^(X)). Thus, we can define a point process by 



det(l + L) 

Let us call it the determinantal point process determined by the operator L.^ 

Proposition 2.1. Let L satisfy the above assumption, 7r(-) be the corresponding 
point process, and p(-) be the associated function as defined above. Set K = L(l + 
L)-\ Then 

p{X) = det Kx . 

Proof. We shall reproduce the argument indicated in [DVJ]. Let f{x) be a function 
on X such that fo{x) = f{x) — 1 is finitely supported. For any point process 7r(-), 

E^(^) n f^y) = E^(^) 11(1 + = 5]p(x) n /o(^)- 

Y yeY Y yeY X xeX 

When the process is defined by an operator L then, identifying / with the diagonal 
matrix with diagonal entries /(a;), we get 

E ^(^) n /(^) = ( E Ly \{f{y)\ det-^ (1 + L) 

Y yeY \ Y yeY J 

= det(l + /L)det"^(l + L) = det ((1 + /L)(l + L)"^) 

= det((l + L + /oL)(l + L)-i) =det(l + /oK) = EdetKx J] /o(a^)- 

X xex 



^This term is not a conventional one. Such processes, not necessarily on discrete spaces, arise 
in different topics, in particular, in connection with random matrices. In [DVJ], they are called 
'fermion processes' but in the the random matrix literature no special term is adopted. 
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Thus, 

j]p(x) n Mx) = 5]detKx n /o(^) 

X xex X xex 

for any finitely supported function /o, which implies p{X) — deti^x- D 

Let pn be the restriction of p(-) to the n-point configurations. One can view pn 
as a symmetric function in n variables, 

Pn{xi, . . . , Xn) = p{{xi, . . . , Xn}), Xi, . . . , Xn pairwise distinct. 

In this notation, the result of Proposition 2.1 reads as follows 

Pn{xi, ...,Xn)= det[K{Xi, Xj)]i<ij<n . 

We call Pn the n-point correlation function. 

From now on we assume that 3£ = X"*" U jC~ (disjoint union of two countable 
sets) and we write £^{X) = £'^{X'^) ® £'^{X~). According to this decomposition we 
write operators in £'^{X) as 2 x 2 operator matrices. 



K 



K. 
K. 



++ 

+ 



K. 



Given a configuration X, we set = X n X^. Wc shall deal with operators L 



\x- 



such that = 0, L = 0. Then, as is easily seen, 7r(X) = unless l^"*"! 

Proposition 2.2. The transforms L K = L{l-\-L)~^ and K h->- L = K{l—K)~^ 
define a bijective correspondence between 
(i) the operators L of the form 

A' 
-B 



L = 



(2.1) 



where the matrix 1 + AB is invertible (equivalently, 1 + BA is invertible) 
and 

(ii) the operators K of the form 

CD C ' 

BCD -D DC 



K 



(2.2) 



where 1 — CD is invertible (equivalently, 1 — DC is invertible). 
In terms of the blocks, this correspondence takes the form 

C={l + AB)-^A = A{l + BA)-\ D = B, 
A = C{1- DC)-^ = {1-CD)-^C, B = D. 

In particular, 

1-CD = {1 + AB)-^, 1-DC ={1 + BA)-^. 

Proof. The proof is straightforward, see [P.V, Prop. 2.2]. □ 
Proposition 2.3. Let K be a J-Hermitian"^ kernel of the form (2.2). Then 

D* 



(2.3) 
(2.4) 

(2.5) 



L = 



-D 



Proof. By Proposition 2.2, L is given by the formula (2.1) with B = D. Since K is 
J-Hermitian, L is J-Hermitian, too. This implies A = B* = D*. □ 

^I.e., Hermitian with respect to the indefinite inner product determined by the matrix J = 

1 







-1 



§3. Calculation of the correlation 
FUNCTIONS. The hypergeometric kernel 



In this section we shall apply the formalism of §2 to the point processes Vz,z',i 
introduced at the end of 51. 



Naturally, we specify X = Z' and X- 



Let us introduce 2 meromoprhic functions in u depending on z' ^ ^ as param- 
eters 



^±W-t ^ [i U T{l±z)T{l±z')T(u + l)T(u + l 



\u + l)T{u + l) 



^ 1/2 .u+i/2 _ t)±{z+z') T{u + l±z)T{u + l±z') 

^ ^ T{±z)r{±z')T{u + l)T{u + l)' 



(3.1) 



An important fact is that the functions i(j±{u) have exponential decay as u tends 
to +00 along the real axis. (Indeed, since ^ G (0, 1), the factor has exponential 
decay; as for the remaining expression in (3.1), it behaves as a constant times 
y^±{z+z ) ^ that it has at most polynomial growth.) 

We shall consider two diagonal matrices of format Z+ x Z+, depending on the 
parameters z,z',^ and denoted as and a third matrix of the same format 
denoted as W: 



= diag 



k\ k\ 



W{k, I) = 



1 



k + l + 1 ' 
(3.2) 

where k,l range over Z+. Note that the A;th diagonal entry of equals ip±{k). 
As the diagonal entries of ^± are real and positive, we may introduce the diagonal 

1 /2 

matrices which are real and positive, too. Note also that W is real and 

symmetric. 

Proposition 3.1. The point process Vz,z',(, is a determinantal process in the sense 
of §2, and the corresponding operator L is as follows: 



L = 







(3.3) 



Note that L is real and J-symmetric. 



Proof. Let A be a Young diagram and X be the corresponding configuration. We 
must prove that 

\ det Lx ,v 
Mzz'£(X) = —- — (3.4) 
^'^'^^ ' det(l + L) ^ ' 

where prime means transposition, we have, taking 



Since L has the form 



A 
-A' 



account of the exact expression for the matrix A (see (3.3)): 

d 

detLx = det^[^(pj,gj)]i<jj<d = JJ V'+(Pi)V'-(Q'i) • det' 
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Pi + Qj + 1 



In the latter product, the factor (1 — ^)^+^' coming from i/j^ cancels with the factor 
(1 - \ and we get 



detLx = det(l + L)-^,e*=i 



X 



n 

i=l 



{z + l)p,{z' + l)p^{-z + l),,{-z' + l) 



PiWQV-Q 



.1 



det' 



1 



Pi + Qj + 1 



Comparing this with the expression (1.1) for M2^^/^^(A), we see that they coincide 
up to a constant factor which does not depend on A. Since the both expressions 
define probability distributions, we conclude that they are identical. □ 

Remark 3.2. As a by-product of the proof we get the following result: 
det(l + L)= det(l + *y V*_W^*y^) = (1 - 0"* • 



By Propositions 3.1 and 2.1, the correlation functions of the point process Vz,z',£, 
are given by the determinantal formula involving the operator K — L{1 + L)~^ . 
Theorem 3.3 below provides an explicit expression for this operator. 

Introduce the functions 

R±{u) = i^±{u) F{^z, Tz'; u+1;-^), (3.5) 

' 1-C u + 1 ' ^ ' 

P±{u) = (V'±(«))-'/'i?±(«), Q±{u) = (V'±(«))"'/'-5±(«). (3.7) 

They are all well-defined for w > 0, because ip±{u) is strictly positive for it > 0. In 

particular, they are well-defined at the points u = k E 

Note that the hypergeometric function that enters (3.5) or (3.6) remains bounded 
as u ^ +00. Hence, the exponential decay of 'iIj±{u) implies the exponential decay 
of R±{u) and S±{u) as u tends to +oo along the real axis. 



Theorem 3.3. Let K = 

the blocks 



K++{k,l) 
K+-{k,l) 
K-+{k,l) = 
K__{k,l) 



be the operator in £ ©£^(Z+) with 

P+{k)Q+il)-Q+{k)P+{l) 
k-l 

P+{k)P-{l)+Q+{k)Q-{l) 
k + l + 1 

P^{k)P+{l)+Q-{k)Q+{l) 
k + l + 1 

P-{k)Q-{l)-Q-{k)P-{l) 
k-l 



Here the functions P±{u) and Q±{u) are defined in (3.7), and the expressions 
K±{k,l) |^_^ are understood according to the L'Hospital rule. 
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Then K = L(l + L) ^ , where L is defined in (3.3). 

We shall call the kernel K defined above the hypergeometric kernel. 
Note that K is J-symmetric (because of the minus sign in the expression for 
K-+). 

Proof. We shall prove that K has the form (2.2) with 

C = K+_ , D = -L_+ = *y V*y^ . (3.8) 

I.e., 

K++ = CD, = DC, K_+ = DCD - D. (3.9) 

As K is J-symmetric, the desired result will follow from Proposition 2.3.* 

It is convenient to slightly rewrite the desired relations (3.9) in order to avoid 
square roots. To do this, we set 

N = *^/^C*i/^ = . 

I.e., 

^ R+{k)R.{l) + S+{k)S_{l) 
^ ' ' k + l + 1 

By virtue of the connection between P±,Q± and R±,S±, the relations (3.9) are 
equivalent to the following ones: 

(W)(M) = -^^±^^^^44i^^^. (3.10) 

ip+{l) k — I 

(VVA^)(,.0^^ ^-W^-('^.-fW^-(') . (3,1) 

We note once more that, by agreement, the indeterminacy arising in (3.10) and 
(3.11) for k = I is removed by making use of the L'Hospital rule. 

To prove the relations above we shall need certain identities involving the hy- 
pergeometric function. 

Lemma 3.4. Set 

Then the series absolutely converge for u ^ —1, —2, . . . and the following relations 
hold 

R±(u) =ip^(u)-^ S^(u), S±(u) = l-ip^(u)-^ R^{u). (3.14) 



^In the same way, one could verify directly that the operators C, D obey the relations (2.3) 
which means that K coincides with L{1 + L)~^. However, reference to Proposition 2.3 makes this 
verification redundant. 
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Lemma 3.5. 



R+{u)R^{-u - 1) + S+{u)S-{-u - 1) = ip+{u)'il;-{-u - 1). (3.15) 

The proofs of these two lemmas can be found in the Appendix (§7). 
Let us now check (3.10). By the definition of N and W, 



Assume first k ^ I. We write 

1 



{k + j + + 1 + 1) k-l \l+j + l k + j + 1 
and plug this into (3.16). Then we get 

{NW}[k,l)- + f^_i 2^^l + j + l 

R+{k) ^ R.jj) _ S+{kl ^ S_ij) 



(3.17) 



k-l ^ k + j + 1 k-l ^ k + j + 1 
By (3.13), this can be written as 

iNW)ik,l) = ^+W^-(0 + ^+W^-(0 _ R+{k)R-ik) + S+{k)S_{k) 

Applying (3.14), we get 

R+ik)S+{l) - S+{k)R+{l) 



R+{k)R_{l) + S+{k)S-{l) 
R+{k)R-{k) + S+{k)S-{k) = 



Ml) 

R+{k)S+{k) - S+{k)R+{k) 



Thus, we have checked (3.10) for /c 7^ 

To extend the argument above to the case k = I, we replace (3.17) by a slightly 
more complicated expression that makes sense for any k,l & Z^: 

nm,|^f-^-^n, (3.18) 



{k + j + l){j + 1 + 1) u^i{k-u\u + j + l k + j + 1 

where u is assumed to be nonintegral. Since the functions R±{u), S±{u) have 
exponential decay as m — > +00 (see the paragraph before Theorem 3.3), we may 
interchange summation and the limit transition. Then we can repeat all the trans- 
formations. At the very end we must pass to the limit as u I, which means that 
we follow the L'Hospital rule. This concludes the proof of (3.16). 

The proof of (3.11) is quite similar, and we proceed to the proof of (3.12). 
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By virtue of the expression (3.10) for NW and our agreement about the L'Hospi- 
tal rule, we get 



{WNW){K I) = lim f: R^U)S-,{u)-S^m^i^ 
Using the transformation 



{k + j + l){j-u) k + u + l\k + j + l {-u-l)+j + l 



we rewrite the above sum as foUows: 



R+ij)S+{u) - S+{j)R+iu) 



^ iP+{u){k + j + l){j-u) 



R+{k)S+{u) - S+{k)R+{u) R+{-u - l)S+{u) - S+{ 



k + u + 1 



+ 



-u 



l)R+{u) 



k + u+1 



Next, applying (3.14), we transform this to 



S-{k)S+{u) + R-{k)R+{u) R+{u) 
il;-{k){k + u + l) k + u + 1 

S-{-u - l)S+{u) + R_{~u - l)R+{u) R+{u) 

ip-{-u-l){k + u + l) k + u + 1' 

Here the second and the fourth fractions cancel each other, while the third fraction 
equals 'ip+{u)/{k + u + 1), because of (3.15). Consequently, the whole expression is 
equal to 

_ S^{k)S+{u) + R-{k)R+{u) tl;+{u) 

ip-{k){k + u + 1) k + u + 1' 

Substituting this expression instead of the sum in (3.19), we get 



{WNW){k,l) = lim|- 



S-{k)S+{u) + R-{k)R+{u) 1 
'0-(A;)'0+('u)(/c + w + 1) k + u + 1 

S-{k)S+{l) +R-{k)R+{l) ^ 1 



i;-{k)i;+{l){k + l + l) k + l + 1 
Thus, 

^ ' ^ i/;_{k)il;+{l){k + l + l) 

which proves (3.12). 

This completes the proof of the theorem. □ 
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§4 Connection with Meixner polynomials 



In this section we shall show that when one of the parameters z' becomes an 
integer, the '++'-block of the hypergeometric kernel defined in Theorem 3.3 turns 
into the Christoffel-Darboux kernel for Meixner orthogonal polynomials. 

The Meixner polynomials form a system {^^{k^a + 1,0} of orthogonal poly- 
nomials, which corresponds to the following weight function on Z_|_: 

fik) - f Ak) - (^ + ^)^^' - ^(^+1 + ^)^' k G 

/ [k) - U,df^) - - - + ' ^ ^ ^+ • 

Here k G is the argument and a > —1 and < ^ < 1 are parameters; 
degd}ln{k;a + 1,0 = For a detailed information about these polynomials see 
[NSU], [KS].9 

Meixner polynomials can be expressed through the Gauss hypergeometric func- 
tion: 

m^{k; a + 1,0 = F{-n, -k; a + 1; ^) 

-Za)[~J^) F{-n,-a-n;l + k-n;^). 



T{l + k-n)T{ 
Basic constants related to these polynomials have the form 

1-^ 



Tln{k; a + = ank'^ + {lower degree terms in A;}, an = 



e ; (« + l)r 



K = I |9Jln(/c; « + 1, Oil' = E ^ + 1' Ofik) - _ + • 

Consider the A^th Christoffel-Darboiix kernel for the Meixner polynomials. It 
projects the Hilbert space / (•)) on the A'"-dimensional subspace spanned by 

the polynomials of degree < N — 1. Let us pass from £^(Z+, /(•)) to the ordinary 
£^ space on Z_|_, which corresponds to the counting measure. Then the Christoffel- 
Darboux kernel will be transformed to a certain kernel, which will be called the 
Meixner kernel and denoted as M^^k, I). We have: 



X 



0- - I L — ly ■ ~ i \ — _L 

ajtjv(fc; a + 1, OaJtjv-i(/; g + 1, - ^N-i{k; a + 1, QMNjl; a + 1, Q 

k-l 



Proposition 4.1. Let z = N + a, z' = N, and let K++{k,l) be the "++" block of 
the corresponding hypergeometric kernel. Then 

K++{k,l) = MN{k + NJ + N). 



^Our normalization of the Meixner polynomials coincides with that of [KS] and slightly differs 
from that of [NSU]. 
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Proof. The proof is straightforward. □ 

Consider the A'"-point "Meixner ensemble" on Z+ whose joint probabihty distri- 
bution has the form 

n 

p{ki,...,kN) = const- Yl (^i - ^j)^II/a,e(^^)• 
l<^<j<Af i=l 

The standard argument due to Dyson (see [Dy], [Me]) shows that the correlation 
functions of this ensemble are given by determinantal formulas with the Meixner 
kernel: 

pn{Xi,..., Xn) = det[MN{ki, kj)]ij^i. 

Then Proposition 4.1 shows that our point process 'Pz,z',£, restricted to the pos- 
itive copy of for z = N + a, z' = N coincides with the trace of the A'"-point 
Meixner ensemble on the set {A'^+l, . . . }. In this subset the number of points 

of the Meixner ensemble can vary from to N, which agrees with our picture. 

§5. Scaling limit of the hypergeometric 

KERNEL: THE WHITTAKER KERNEL 

Recall that the construction of the point processes Vz,z',c was started from cer- 
tain probability distributions on partitions of an integer number n denoted as M^^l , 
see §1. These distributions possess an additional important property: they con- 
verge, as n — cxo, to a probability distribution on a certain limit object Q called 
the Thoma simplex: 

CO 

1^ = {ai > a2 > . . . > 0; A > /32 > . . . > I ^(a^ + (3i) < 1} . (5.1) 

It is a compact topological space with respect to the topology of coordinate-wise 
convergence. 

More precisely, for every n we embed the set Y„ of partitions of n into fl by 
making use of the map 

Yn3 X = {pi,...,Pd\qi,---,qd) 
f pi + 1/2 Pd + 1/2 ^ ^ qi + 1/2 gd + 1/2 \ (5.2) 

I— > I ) • • • ) ) u, U, . . . , ) • • • ) , U, U, . . . 1 . 

\ n n n n J 

Next, we identify M^"'^, with its push-forward under the map (5.2), so that M^^^l 
turns into a probability measure on Q with finite support. 

(n) 

Theorem 5.1. The measures J, weakly converge to a probability measure Pz,z' 
on Q as n ^ oo. 

Proof. This follows from a general theorem, see [KOV]. □ 

Recall now that to construct the process Vz,z' ,i on the lattice Z' we have mixed 

(n) 

all the distributions , n = 0, 1, 2, . . . , using the negative binomial distribution 
with suitable parameters, see §1: 

7r(n) = (1 - 0* % r , ee(0,l). (5.3) 

it* 
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Let us embed U into the punctured line R* = R \ {0} and then rescale the process 
'Pz,z',£, by multiplying the coordinates of its points by (1 — ^). Then the rcscalcd 
point configuration in R* that corresponds to A G Y„ differs from the image (5.2) 
of A in O by the scaling factor (1 — ^)n. 

The discrete distribution on the positive semiaxis with 

Prob{(l - On} = (1 - 0* n = 0, 1, 2, . . . 

which depends on the parameter ^ e (0, 1), converges, as ^ — > 1, to the gamma- 
distribution with parameter t 

lids) = ^ e-^ds. (5.4) 

This brings us to the following construction. Consider the space Q, = fl x R+ 
with the probability measure 

Pz,z' = Pz,z' ® ^ e-'ds. 

To any point (a; = (q;|/?), s) e Q we associate a point configuration in R as follows 
{{a\P),s) ^ (ais, a2S, -Pis, -/?2S, . . . ). (5.5) 
Thus, the measure Pz,z' defines a point process on R* which will be denoted as 

Vz,z' 

Then the considerations above together with Theorem 5.1 suggest the following 

Theorem 5.2. The point processes Vz,z',^ scaled by (1 — ^) converge, as ^ ^ 1, to 
the point process Vz,z'- 

We will give a rigorous formulation of this claim and its proof in our next paper. 
Meanwhile, we will use this theorem as a prompt. 

The main result of our previous work [P.I] - [P.V] was an explicit computation 
of the correlation functions of Pz,z'-^^ To formulate this result we shall need the 
classical Whittaker function Wk,ij.{x), a; > 0. 

This function can be characterized as the only solution of the Whittaker equation 

such that y ~ x'^e~i as a; ^ +oo (see [El, Chapter 6]). Here n and /j, are complex 
parameters. Note that 

We shall employ the Whittaker function for real k and real or pure imaginary fx; 
then VFk,/^ is real. 

We introduce the functions 



^^^^^ = (r(l ± z)Y{l ± z') x)V2 (^)' ^ ^ 



' 2 



About correlation functions of point processes living on a nondiscrete space, see [DVJ], [LI], 

[L2]. 
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Theorem 5.3. The correlation functions of the process Vzz' have the form 

n = 1,2, . . . ; til, . . . e E*, 
where the kernel )C{u, v) is conveniently written in the block form 



/C(w, v) = < 



/C++(«,f), u,v > 0; 

/C-i {u,—v), u>0,v<0; 

/C |_(— u<0,v>0; 

^ K. {—u,—v), u,v<0; 



with 



f^-+{x,y) = - 
!^—{x,y) = 



V+{x)Q+{y) - Q+{x)V+{y) ^ 
x-y 

V+{x)V-{y) + Q+{x)Q-{y) ^ 
x + y 

V_{x)V+{y) + Q_{x)Q+{y) 
x + y 

V-{x)Q_{y) - Q_{x)V-{y) 



x-y 

The kernel lC{u,v) is called the Whittaker kernel, see [P.IV, Th. 2.7], [BOl, Th. 
Ill], 11 

Clearly, the hypergeonietric kernel (see Theorem 3.3) and the Whittaker kernel 
have the same structure. Theorem 5.2 prompts that the Whittaker kernel is the 
scaling limit of the hypergeometric one. In the next theorem we establish this fact 
by a direct computation. 

Theorem 5.4. For the hypergeometric kernel K given by Theorem 3.3 and the 
Whittaker kernel K, given by Theorem 5. 3 the following limit relation holds 



1 



lim 



( 


u 




V 






1 





= K.**{u, v), 



where the subscript ** stands for any of the four symbols ++, H — , — h, • 

Proof. Take x,y > and denote 



k = 



X 



1-e 



I = 



y 



Then (1 - ^)k x, {1 - ^)l ^ y. Since 

1 



1-e 



k—l x—y^ k+l+1 x+y^ 



^^In that papers, the term 'Whittaker kernel' concerned the block while the kernel K, was 
called the matrix Whittaker kernel. 
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it is enough to show that 

P±{k) ^ V±{x), Q±{k) ^ Q±(x). (5.7) 

We shall employ the following asymptotic relation which connects the hypergeo- 
metric function and the Whittaker function: 

lim F(a, 6; 1 - -) = a;°^2 ^ e^W -a-b+i a-b (x), x > 0, (5.8) 

U^+OO ^ 2 ' 2 

see [El, 6.8(1)]. Note that = 1 — Applying (5.8) we get the following limit 
relations for the hypergeometric functions entering (3.5) and (3.6): 

^ 2 ' 2 

F{lTz,lTz';k + 2;-^) ^^^^ 



(1-0(^ + 1) 



2 ' 2 



Next, the factor {i/^±{u))^/'^ entering (3.7) behaves as follows {i/^±{u) was defined in 
(3.1)): 



1/2 



r(i±^)r(i±^') 

Finally, from (3.7) we obtain 

^1/4 

{T\l ± z)T{l ± z')x)^^'^ 2 ' 2 

^3/4 

^^^^^ " (r(i ± zni ± z')xy/^ = ° 



Remark 5.5. As was demonstrated in §4, the "++"-block of the hypergeometric 
kernel turns into the Christoffel-Darboux kernel for Meixner polynomials when 
z = N + a, z' = N,Ne Z_|_. It is well known that in the scaling limit as ^ — 1, the 
Meixner polynomials turn into the Laguerre polynomials (see [KS], [NSU]). This 
agrees with the fact that for z — N + a, z' — N, the restriction of the process 'Pz,z' 
to the positive semiaxis coincides with the A'"-point Laguerre ensemble, see [P.III, 
Remark 2.4]. 

Note that the shift by which we were doing to match Vz,z',£, and the Meixner 
ensemble disappears after we take the limit. 

Remark 5.6. A straightforward check shows that the scaling limit of the kernel 
L(x, y) defined by (3.3) is the kernel £(x, y) of the operator C = /C(l — /C)~^ where 
K, is the integral operator in L^(R*, dx) corresponding to the Whittaker kernel (the 
kernel jC{x,y) was explicitly computed in [P.V, Theorem 2.4]). 
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§6. Integrable operators 



In this section we shall show that the operator given by the Whittaker kernel 
belongs to the class of integrable operators as defined by Its, Izergin, Korepin 
and Slavnov [IIKS]. We shall also argue that the hypergeometric kernel might be 
considered as an example of a discrete kernel giving an 'integrable operator'. 

We shall follow [De] in our description of integrable operators. 

Let S be an oriented contour in C. We call an operator V acting in L^(S, \d(\) 
integrable if its kernel has the form 

nC,C')= ^^"'/y'^'' , C, C'es, 

for some functions fj, gj, j = 1, . . . , N. We shall always assume that 

N 

so that the kernel V^(C) CO is nonsingular (this assumption is not necessary for the 
general theory). 

The notion of an integrable operator was first introduced in [IIKS] . 
It turns out that for an integrable operator V the operator R = V{l + V)~^ is 
also integrable. 

Proposition 6.1 [IIKS]. Let V be an integrable operator as described above and 
R=l-{1 + V)-^ = V{1 + V)-^. Then the kernel R{C, (') has the form 

where 

F, = (l + y)-V,-, Gj = {l + V'r'gj, j = l,...,N. 

^/Ef=i fjiOgjiO = on S, then Ef=i Fj{C)GjiC) = Oon^as well. 
Proof. See [KBI, ch. XIV], [De]. □ 

It is not difficult to show that for integrable operators Vi, V2, the product V1V2 
is also integrable. This fact and Proposition 6.1 imply that operators of the form 
I + V where V is integrable form a group. 

A remarkable fact is that the function Fj, Gj can be expressed via a suitable 
Riemann-Hilbert problem, see [IIKS], [De] for details. 

Now we pass to a much more special situation. Let E = M*. According to the 
splitting M* = M_(_ U M_ and further identification of M_ with a second copy of , 
we shall sometimes write the kernels of operators in L^(]R*) in block form. 

Consider an integral operator V on R* whose kernel V{x,y) has the following 
block form 



[V]{x,y) = 



x+y 

h+{y)h-{x) Q 
x+y 
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x>0,y>0, (6.1) 



for some functfons h+{x) and h-{x) defined on the positive semiaxis. Then the 
operator V is integrable. Indeed, 

x-y 

where 

f 0, a; > f /i+(a;), x > 

I, a;), a;<0 1^0, x <\) 

r X > r 0, X > 

0, X < [ x), X < 

Assume that there exist four functions A±{x), B±{x) defined on the positive 
semiaxis such that 

where 

^ (6.4) 

Jy>Q a; -|- y 

is the Stieltjes transform. 

Then Proposition 6.1 imphes that the kernel of the operator R = V{1 + V)~^ 
has the form 

X-y 

with ^ ^ ^ ^ ^ 

-^+(^) a; > r ^+(^) a; > 

In bfock form the kernel R{x,y) can be written as follows: 



[R]{x,y) 



1 A+(x)B+(y)-B+(x)/l+(y) 1 A+{x)A_{y)+B+{x)B_{y) 

h+{x)h+{y) x-y h+{x)h-{y) x+y 

1 -A_{x)A+{y)-B_{x)B+{y) i A_ (a;)B- (y)-B- (a;)A_ (;/) 

h-(x)h+(y) x+y h-{x)h-{y) x-y 



All these formulas work perfectly well for the Whittaker kernel. If we set, using 
the notation of §5, 



h±{x) = x^^~e 

TT 

A±{x) = h±{x)V±{x), B±{x) = h±{x)Q±{x), 

then the kernel R{x, y) coincides with the Whittaker kernel K,{x, y). The form (6.1) 
of the kernel of y = /C(l — 1C)~^ was obtained in [P.V]. The formulas (6.3) in this 
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case can be derived from the known formulas for the Stieltjes transform of the 
(suitably normalized) Whittaker function [E2, 14.3(53)]. 

It is a remarkable fact that all the formulas above also work for the hypergco- 
metric kernel. This kernel lives on the lattice Z' x Z', so one can call the operator 
corresponding to the hypergeometric kernel a discrete integrable operator. 

Indeed, exact expressions for h+{k) and h-{k) can be easily extracted from (3.3) 
if we take V = L: 

h±{k) = {^±{k)f'\ 
Then, as before, we have, cf. (3.5)-(3.7), 

A±{k) = h±{k)P±{k) = R±ik), B±{k) = h±{k)Q±{k) = S±ik), 

and the kernel R{x,y) coincides with the hypergeometric kernel K{x,y). 

Relations (6.3) are exactly the relations of Lemma 3.4, see (3.14). 

If we consider the continuous case, then from the general theory of Riemann- 
Hilbcrt problems one can extract the following identity for the analytic continua- 
tions of B±, A±: 

A+{OA_{-0 + 5+(C)5_(-C) = hliOhU-0, C e C \ R. (6.5) 

It means that the determinant of the solution of the corresponding Riemann- 
Hilbert problem is identically equal to 1 (this follows from the fact that the deter- 
minant of the corresponding jump matrix is identically equal to 1). 

Though we do not have an analog of the Riemann-Hilbert problem in the discrete 
case, a discrete analog of (6.5) still holds, see Lemma 3.5. 

Remark 6.2. Both the Whittaker and the hypergeometric kernel possess the sym- 
metry property 

R{x, y) = sgn(x) sgii{y)R{y, x) (6.6) 

which, perhaps, emerged for the first time: most (integrable) kernels arising in 
Random Matrix Theory and mathematical physics are simply symmetric. The 
formula (6.6) means that the corresponding integral operator in L'^(M.,dx) or in 
£^(Z') is symmetric with respect to an indefinite inner product. 

§7. Appendix: some relations for the Gauss hypergeometric function 

In this section we prove Lemmas 3.4 and 3.5. First, we shall reformulate them 
using (3.5) and (3.6). 

Lemma 3.4'. For ^ e (0, 1) the following decompositions hold: 

F(a, 6; + 1; ^) _ - (a),(6).e^(l - 0^+^'^ h- k + 1- ^ ) 
u k\k\iu + k) F{l-a,l-b,k + l,^^), 

1 - F(a, 5, u, = X. k\k\{u + k) • 

Specifically, the series in the RHS absolutely converges for w 7^ 0, —1, —2, . . . and 
represents a meromorphic function; the both formulas are viewed as equalities of 
meromorphic functions in u. 



21 



Lemma 3.5'. 



F{—z, —z'; u + 1; w)F{z, z \ —u; w) 

, , . F{~z + l,-z' + l;u + 2;w) F{z + l,z' + l;-u + l;w) 

+zz w{l — w) = 1. 

u + 1 u 

(7.1) 

Proof of Lemma 3-4 '■ Let us check the first relation. The RHS has the form 

u + k' 

k=0 

where the coefficients Ak rapidly decrease as A; — > oo, because of the factor (the 
factor F{l-a,l-b;k + 2; ^) remains bounded as A; — > +cxo, and the remaining 
expression has at most polynomial growth in k). Consequently, the RHS is indeed 
a converging series representing a meromorphic function in u. This function has 
simple poles at'u = 0,— 1,— 2, Using the formula 

Res F{a, b; c; w) = Fia + k + l,b + k + 1; k + 2;w) 

c=-k {k + l)\k\ 

{a)k+i{b)k+ii-l)''w^+\l - w)—'-'^ 
= ^:p^y|^^ F{l-a,l-b,k + 2,w) 

(7.2) 

for the residues of the hypergeometric function, one verifies that the residues of the 
RHS at tt = —1, —2, . . . are the same as for the LHS, and it is directly seen that 
the residues at m = coincide, too. Moreover, the same claim holds not only for 
^ G (0, 1) but for any complex ^ ranging over the unit disc |^| < 1, and the both 
sides are holomorphic in ^. 

Let us expand both the LHS and the RHS into Taylor series in ^ and compare 
the respective Taylor coefficients. Each Taylor coefficient (on the left and on the 
right), viewed as a function in a,b,u, is a rational expression which is polynomial 
in a, b. This implies that it suffices to prove our relation, say, for a — 0, —1, —2, . . . . 

Thus, we may assume that a = 0, —1, —2, For these values of a, the LHS 

becomes a rational function in u, and the series in the RHS terminates and, con- 
sequently, is a rational function in u, too. Next, we know that the both sides have 
the same singularities. Finally, they both behave as 0{l/u) as \u\ oo (indeed, 
as was mentioned above, the hypergeometric function in the numerator of the LHS 
is 1 + 0(1/m), so that the whole expression is 0(1/^), and for the RHS the same 
holds, because the series terminates). Consequently, the both sides are identical. 
This concludes the proof of our first relation. 

The second relation is verified similarly. □ 

Proof of Lemma 3. 5 '. We use the same argument as in the previous proof. The 
desired relation can be viewed as an equality of power series in the variable w. The 
coefficients of the series are polynomials in z^z' , so that we may assume, without 
loss of generality, that one of the parameters z, z' takes positive integer values while 
another takes negative integer values. In this case all the four hypergeometric series 
entering our relation terminate and, so, are rational functions in u. 

Next, we examine the possible singularities of the LHS of (7.1). Here only simple 
poles at the points w e Z may occur, but it turns out that the residue at any w e Z 
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vanishes. That is, the contributions of the two products cancel each other. To see 
this, one can use, for example, (7.2). 

Finally, we remark that, under our specialization of z^z' ^ the LHS of (7.1) is 
1 + 0(1/ w) as \u\ ^00 . This concludes the proof. □ 
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